DILEMMA-2: A Lemmatizer-Tagger For Medical Abstracts

This paper reports on the development of DILEMMA-2*, a lemmatizer-tagger for the sublanguage of medical abstracts. The program is an extension of DILEMMA-I, a lemmatizertagger for general English texts. In the first section a brief outline is given of DILEMMA-1. Particular attention _is paid to the original concept of a default category which is linked with a categorial graph by means of a pointer system. In the second section we show why DILEMMA-1 was not able to get a suitable score when lemmatizing medical abstracts, the main reason being the inability to recognize sublanguage specific vocabulary. In the next section a description is given of the most important errors along with their solutions; these errors are then categorized as gaps or wrong assignments. The former could be dealt with in either a suffix list or a gaps filler default. The latter mainly concerned wrongly assigned past participles and errors on noun, verb or adjective assignment. After implementation of the proposed solutions, a comparison is made between the results of DILEMMA-1 and DILEMMA-2, showing that the results of DILEMMA-1 have been improved substantially within a sublanguage context, and this by using linguistic, i.e. sublanguage, knowledge, thus avoiding ad hoc remedies. DILEMMA-2 was developed as part of a research contract for Elsevier Science Publishers (ESP), Amsterdam, The Netherlands. The development of DILEMMA-lwas carried out as part of contract research for Van Dale Lexicografie Publishers, Utrecht, The Netherlands. In this paper we describe DILEMMA-2, a lemmatizertagger for medical abstracts, which is an updated version of DILEMMA-1, a lemmatizer-tagger for general texts. After a brief outline of DILEMMA-1 we give a description of the types of errors we found when running the general lemmatizer on medical abstracts. This is followed by some examples of the solutions we proposed and implemented into DILEMMA-2. Finally, the results of DILEMMA-I and DILEMMA-2 are compared, showing that a sublanguage approach can lead to workable results in the development of real world applications.